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DESCRIPTION 

IMAGE PROCESSING APPARATUS, IMAGE PROCESSING METHOD, AND 
RECORDING MEDIUM 

Technical Field 



processing method, and a recording medium, and more specifically to an image 
processing apparatus, an image processing method, and a recording medium which are 
suitable for the case of synthesizing a hair image expressed by computer graphics .with . 
a three-dimensional shape image of a head part is removing a hair. 

Background Art 

Conventionally, as a method for expressing human.hair by computer graphics '■ 
(hereinafter abbreviated as CG), it is usefiil to take a method in Which an entire hair 
style is approximated to a polygon model, as shown in FIG. 1 A, and a texture of hairs ; , 
is adhered to the model, or a method in which rendering is performed with use of an 
anisotropic reflection model which has different reflection intensities respectively with . 
respect to light source positions or viewpoint positions. . . 

These methods are described in "Hair expression using an anisotropic reflection 
model" by Yamana and Suenaga, Singakugihou, PRU87-3, pp. 115 to 120, 1989. In 
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these methods, expression of a hair style is just the shape of a polygon model itself, 
and the hair style can be changed by changing the shape of the polygon model is 
realized. 

A proposal has been made for a method of expressing hair by a particle system. 
In the particle system, when a large quantity of particle objects are dealt with, 
Li, characteristics such as weight and the like of each particle are given as parameters, and 
Cj. particles are generated under a certain rule. Thus, a large quantity of particle objects 
M can be expressed easily. 

ft : Meanwhile, the present apphcant has proposed a method of approximating 

. . g every one.of hairs to a curve, as shown in FIG. 1 B, in Japanese Patent Application No. 
u 10-164545, for example. This method is successful in natural expression of a hair 

yi ' 

C3 style image. 

Hi 

A method of designing a hair style by editing together a plurality of hairs 
. . approximated.to a curve is described in "Development of a hair design system using 
^ tuft models" by Kishi, Saegusa, and Morishima, in Shingakugihou, MVE97-1G2, pp. 
67-74, 1998. : . 

It is difficult for the method of approximating the entire hair style to a polygon 
. model to express a complicated hair style. In this method, movements (flows) of hair 
. cannot be expressed. 

On the other side, the method of expressing hair by a particle system and the 
method of approximating every one of hairs to a geometric figure can express any 
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arbitrary hair style and movements of hair. However, in the method of expressing hair 
by the particle system, parameters for particles expressing hairs must be controlled to 
control the hair style, so that intuitive control is difficult. It is therefore difficult to 
. generate an intended hair style. 

In the method described in "Development of a hair design system using tuft 
. models" by Kishi, Saegusa,andMorishima,inShingakugihou,MYE97-102,pp. 67t74, 
p .I998,.an intended hair style can be generated relatively easily. This method, however, 

?3 has a problem in that hair cannot be changed to intended hair styles corresponding to 

p 

r.j a plurality of different head shapes although hair corresponding only to one fixed head 

fjv .shape can be changedvto an intended hair style, : . 

m 

Disclosure of the Invention 

nj . ■ 

i ..The present invention has been made in view of the actuailsitoation 

: ^ above and has an object of changing a hair style consisting of hairs ^every of which is 

expressed by GG, so as to correspond to plural of different head shapes. 

The present invention proposed to achieve the above obj ect provides an image 

processing apparatus for synthesizing a hair image with a three-dimensional shape 
. image of a head part, to. generate a hair-style-matched image,, comprising: storage 

means for storing a hair-style data piece; read out means for reading out the hair-style 

data piece stored in the storage means; mapping means for mapping a hair data piece. 

corresponding to a hair and contained in the hair-style data piece read out by the read 
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out means, at a predetermined position on the tliree-dimensional shape image of the 
head part; and generation means for generating the hair-style-matched image, based 
, on a mapping resuU obtained by the mapping means. 

As the hair data piece, it is possible to use three-dimensional curve data which 
. is constructed by a plurality of control points. 

. . . As the hair.data piece,:it is possible to use polygon data which is constructed by 
fj. a.plurahty of polygons. ■ 
B3 The storage means stores the hair-style data piece constructed by plural hair 

J^j data pieces arranged on a two-dimensional array. In particular, the storage means 
Pj stores the hair-style data piece constructed by plural data pieces : arranged on a 
ya, two-dimensional array corresponding to a projected image obtained by projecting a: 
E3 three-dimensional hair style expressed by a columnar coordinate system, on a 
V : two-dimensional coordinate, system. . / - . > . : . . ^ 

The apparatus according to the present invention further comprises i^^ 
means for interpolating the hair data piece at the predetermined position, with use of 
a hair . data piece close to the predetermined position. The interpolation means. . 
, V interpolates the hair, data piece, based on group informationxontained in the hair data 
piece close to the predetermined position. \. . . . 

. . ...The.mapping means. includes coordinate conversion meanS'for convierting the 
. . : hair data piece expressed by a first coordinate system into data of a second coordinate 
system by which the three-dimensional shape image of the head part is expressed. 



The coordinate conversion means converts the hair data piece expressed by the 
first coordinate system into data of a local coordinate system having an origin on a 
surface of the head part, and thereafter converts the data into data of the second 
coordinate system by wliich the three-dimensional shape, image of the head part is 
expressed. 

. When the hair data piece expressed by the first coordinate system is converted 
into the data of the second coordinate system by which the three-dimensional shape 
image of the head part is expressed, the coordinate conversion means executes at least 
one of coordinate axis rotation and origin shift, based on a random number. 

AlsOi the present invention provides an image processing .method for 
synthesizing a hair image with a three-dimensional shape image of a head part, to 
generate a hair-style-matched image, comprising: a storage step of storing a hair-style 
data piece; a read outstep of reading out the hair-style data piece stored by processing 
in the storage step; -a. mapping step of mapping a hair data piece corresponding to a 
hair and contained in the hair-style data piece read out by processing in the read 
. outstep, at a predeteraiined position on the three-dimensional shape image of the head 
party and a generation step of generating the hairr sty le^matched image,, based on a- 
mapping result obtained by processing in the mapping step, 

: . . In the processing in the storage step according to the present invention, the 
hair-style . data, piece constructed by plural hair data pieces arranged on. a 
two-dimensional array is stored. 
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Also, in the processing in the storage step, the hair-style data piece constructed 
by plural data pieces arranged on a two-dimensional array corresponding to a projected 
image obtained by projecting a three-dimensional hair style expressed.by a columnar 
coordinate system, on a twordimensional coordinate system, is stored. 

The method according to the present invention further comprises an 
interpolation step of interpolating the hair data piece at the predetermined position, 
- with use of a hair data piece close to the predeteraiined position. 
53 In processing in the interpolation step, the hair data piece is interpolated, based 

a 

yJ on group infonnation contained in the hair data piece close to the predetennined • 
- position;-. . . 

. i / The^ processing in the mapping step includes a coordinate conversion, step.of* 

|JS| 

Q converting the hair data piece expressed by a first coordinate system into data of a 
second coordinate system by which the three-dimensional shape image of the head part 
i^.is expressed. ..i:/..:^-,- •..-•.v:-.- 

^ In processing in the coordinate conversion step, the hair data piece expressed 
by the first .coordinate system is converted into data of a local coordinate system 
having an: origin on a surface of the head part, and thereafter, the data is converted into 
: data of the second coordinate system by which the three-dimensional shape image of 
. the head part is expressed. 

.When the hair data piece expressed by the first coordinate system is converted 
into the data of the second coordinate system by which the three-dimensional shape 



image of the head part is expressed, at least one of coordinate axis rotation and origin 
shift is executed, based on a random nmnber, in processing in the coordinate 
. conversion step.. 

Fmther, the present invention provides a recording medium. which records a 
computer-readable program for image processing of synthesizing a hair image with a . 
threerdimensional shape image of a head part, to generate a hairrstyle-matched image, 
;the prograin comprising: a storage step of storing a hair-style data piece; a read outstep 
of reading out the hair-style data piece stored by processing in the storage step; a 
mapping step of mapping a hair data piece corresponding to a hair and contained in the 
hair-style data piece read out by processing in the read outsteps at a predetemiined 
position on:the;three-dimensional shape image of the:head part; and a generation step 
of generating the hair-style-matched image, based on a mapping result obtained by 
processing in the rnapping step. .^ -ly^^,-^ - , -;. 

. In the processing in the storage step, the hair-style data piece, constructed by 
plural hair data pieces arranged on a two-dimensional array is stored. — 

: vin the processing in the storage step, the hair-style, data piece constructed by 
pliiral data pieces arranged on a two-dimensional array corresponding to. a projected 
image obtained by projecting a three-dimensional hair style, expressed.by a colmimar 
coordinate system, on a two-dimensional coordinate system,, is stored. 

...According to the present invention, the program further . comprises an 
interpolation step of interpolating the hair data piece at the predetermined position. 
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with use of a hair data piece close to the predetemiined position. 

In processing in the interpolation step, the hair data piece is interpolated, based 
on group information contained in the hair data piece close to the predetermined, 
position.. . .... 

.The processing, in the mapping step includes a coordinate conversion step of 

converting the hair data piece expressed by a first coordinate system into data of a 
second coordinate system by which the three-dimensional shape image of the head part 
is expressed. 

In processing in the coordinate conversion step, the hair data piece expressed 
.,by the; first coordinate system is converted into data bf a. local coordinate, system 
: having an origin on a surface of the head part, and thereafter, the data is converted into 

data of the second coordinate system by which the three-dimensional shape image of 

the head part is expressed. 

: : .When the hair data piece expressed by the first coordinate sysftem is converted 

into the data of the second coordinate system by which the three-dimensional shape 

image of the head part is expressed, at least one of coordinate axis rotation and origin; 

shift : is executed, based on . a random number, in . processing- in. the. coordinate 

conversion step. 

.... In the above-described image processing apparatus and method according to the 
present invention, hair-style data is stored, and the stored hairrstyle: data is read out. 
Hair data corresponding to a hair and included in the read outhair-style data is mapped 



at a predetennined position on a three-dimensional shape image of a head part. Based 
on the mapping results, a hair-style-matched image is generated. The stored hair-style 
data is constructed by plural pieces of hair data arranged on a two-dimensional array. 

Further other objects of the present invention and specific advantages which 
will be obtained from the present invention will be more clearly understood from the 
following description of embodiments. - . . . 

Brief Description of the Drawings 

FIGS. 1 A and IB are views showing an example of conventional hair 
■ expression.. .-. V :. vi--. ...v- ... 

V " FIG. 2 is: a block-diagram showing a structural example of: a hair synthesis 
system to which the present invention is applied. 

FIG. 3 is a flowchart explaining operation of the hair synthesis , system. 
: FIG. .4 is a view for explaining the operation of the hair synthesis sy stem; • : 
FIG. 5 is a block diagram showing a structural example of processing in the 
hairrstyle mapping processing section shown in FIG.:.2. . . . 

.. vFIG. 6 is a flowchart for explaining processing in . the hair-rstyle . mapping 
processing section. 

. . : V FIGS, 7A and 7B are views for explaining hair-style data in which hair data is 
two-dimensionally arrayed. 

FIG. 8 is a view for explaining mapping position calculation in the step S13 
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shown in FIG. 6. 

FIG. 9 is a view for explaining hair data interpolation processing in the step S 1 5 
. shown in FIG. 6. , 
... FIGS. lOA and lOB are views for explaining coordinate conversion in.the step 

S 16 shown in FIG. 6 

FIG. 11 is. a view showing a data structure of the hairrstyle database shown in. 
I FIG. 1. 

B3 FIG. 12 is a view for explaining interpolation processing using group 

C3 

yJ , information included in hair data. . . :. i . 

yj 

FIGS. 13A and 13B are views for explaining coordinate axis rotation in 

01 

coordinate conversion.: r-- -.-y-.... 

m 

E3 FIGS. 14 A and 14B are views for explaining coordinate axis shift in coordinate 

m 

.... conversion....-^.. ■ -.• v. . • . r. . . .... ... 

. . > : PIGS:. J5A and? 15B are views for explaining of mapping to :the three- 
dimensional rhodel of a head part with an arbitrary changed. - 
. : / FIG. 16 is a view showing a colunmar coordinate system when hair-style data 
^.is generated.. • • •V z-^.-.. v.. ;;..,V'. 

FIG. 17 is a view showing a stmctural example of a personal computer which 
realizes an image processing apparatus. . ; 
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A hair synthesis system to which the present invention is applied will be 
explained with reference to FIG. 2. This hair synthesis system comprises: an image 
pickup device 1 which picks up an image of a neck part (the^ part above the neck, 
including the neck) of a person; an image processing device 2, for generating a 
three-rdimensional shape model of the neck part, based on the image of the picked-up 
neck part, , synthesizing a CG of an arbitrary hair, style, . and generating a 
hair-style-matched image; and a display device 9 for displaying the generated 
hair-style-matched image. If this hair synthesis system is installed in a beauty salon, 
a barber's shop, or the like, it can be used to assist customers to select hair styles or so. 

. The image pickup device . 1 of the hair synthesis system combines a 
three-dimensional shape measurement device such as: a stereo camera, a range finder 
or the like, and a two-dimensional image input device such as a CCD (Charge Coupled 
Device) or the like. In case of a stereo camera, the three-dimensional measurement 
device o f the image pickup device 1 picks up. an imaige of the headpart of a person (a 
customer of a beauty salon or the like) as a target object, several times from a different 
kinds direction; and performs predetennined image processing on.the. image, to obtain 
a three-dimensional shape of the face part (which is a part obtained by removing a hair 
part from the neck part). In case of a range finder, the neck part of the person as the 
■ target object is. subjected to laser scanning, thereby measuring.^the distance,. to obtain 
a three-dimensional shape of the neck part. The two-dimensional input device of the 
image pickup device 1 picks up an image of the neck part of the person as the target 
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object, to obtain a texture corresponding to the three-dimensional shape of the face 
part. . 

In general, the three-dimensional shape measurement device of the image 
. pickup device 1 cannot obtain three-dimensional shape data.with respect to a part of. 
. dark color which absorbs hght from an object, e.g., a part such as a head part having 
. black hair or metal which reflects light. Therefore, the image pickup device 1 outputs 
^ . threerdimensional shape data of the part such as the face or the like, which will be 
03 hereinafter called a face part, i.e., the other part of the head part than the hair, and 
bj texture image data associated with the three-dimensional shape data of the face part. 

yj 

8 Plural three-dimensional shape measurement ; devices and . plural 

n 

issr 

V two-rdimensional image input devices may be used for cbnstmcting the image pickup 
p2 device 1, in order to measure the neck part of the person as an object (which will be 

?« 5 

described hereinafter as axustomer) from many directions. • - v - .: f 
I . / : The iiTiage processing device 2 is constructed by: a thf ee-dirnensional head part 

reconstraction processing section 3 for synthesizing three-dimensional shape data read 

from a head-part. database .6 with three-dimensionaLshape data. inputted from the. 
. image pickup device 1 ; to constmct a three-dimensional model. of the neck part; a hair : 
. style mapping processing section 4 for mapping hair-style data read from the hair-style 
. . database 5. over the three-dimensional model of the neck part; a rendering section 5 for 

rendering the three-dimensional model of the neck part on which, the hair-style data . 

has been mapped, thereby to generate hair-style-matched image data; a head-part 
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database 6 in wliich three-dimensional shape data of a plurahty of head parts are 
previously recorded; a hair-style database 7 in which a plurality of hair-style data items 
are previously recorded; and a controller 8 which is operated by a user to instruct 
selection of three-dimensional shape data of a head part stored in the head-part 
database 6, a hair-style data item stored in the hair-style database 7, or the like. 
1.1. The hair-style data stored in the hair-style database 7 is edited and generated 

C3 with use of a three-dimensional graphic modeler or the like. Therefore, even a hair 

03 style which is actually difficult to make up can be generated and stored. The data 

C3 

fi structure of the hair-style database 7 will be described later with reference to FIG. 1 1 . 

Q The display device 9 is constructed by a LCD (Liquid Crystal Display) or CRT 

y I 

1=^ (Cathode Ray Tube) and the like, and displays a hair-style-matched unage mputted 
fi-om the image processing device 2. 

Next, the operation of the hair synthesis system will be explained with reference 
to the flowchart of FIG. 3 and FIG. 4. In the step SI, the image pickup device 1 picks 
up an image of the a neck part of a customer as a target object or laser-scans the neck 
part, to measure its three-dimensional shape, and outputs three-dimensional shape data 
and texture image data of the neck part of the customer to the image processing device 
2. 

The measurement method of measuring the three-dimensional shape may be any 
method, e.g., a method of using a stereo camera, a method of using a range finder, or 
the like. As described previously, however, the three-dimensional shape measurement 



rj 
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device of the image pickup device 1 cannot obtained three-dimensional shape data of 
the hair part.. Therefore, three-dimensional shape data and texture image data (shown 
as measurement data in which a texture image is applied to three-dimensional shape 
data of the face . part in FIG. 4) of the face part of the customer is outputted by the 
. processing in the step S 1 . If plural three-dimensional shape measurement devices and 
two-dimensional image input devices are provided in the. image pickup device 1, a 
plurahty. of sets each including three-dimensional shape data of the face part and 
texture image data are outputted. 

In the step S2, the three-dimensional head part reconstruction processing 
section 2 ofthe image processing device 2 constructs athree-dimensional shape model 
of ahead part of a customer by using measurement data of the face part inputted from : 
the image pickup device 1, i.e., the three-dimensional shape data and texture image 

: Spjeeificallys. as shown in FIG; 4, head-part data selected arid, read from the 
head-part database 6 is combined with the measurement data of the face part of the 
customer; to reconstruct a . perfect threerdimensional model of the head part of the. 
customer. The:head-part database 6 is constructed by three-dimensional models of a 
plurahty of head parts having arbitrary shapes. The head-part data is subjected to 
scaling and positional aligning,. based on the size and direction in .the measurernent. 
dataof the face part of the customer, and is then combined with the measurement data.. 
of the customer. 
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The combination of the head-part data piece with the measurement data piece 
of the customer may generate a new polygon between open edges in both data pieces, 
. or an open edge in either one data piece may be changed to fit an open edge of the 

other .data piece. - . : / . 

. . . The tliree-dimensional shape of the head part of the customer thus constructed 
. is expressed, for example, as a triangle-polygon model. Jn the three-dimensional 

C3 

f J. model of the head part of the customer thus constmcted, hair is removed from the real 

D3 image of the customer as shown in FIG. 4. 

D 

W In the step S3, operation of selecting hair-style data stored in the hair-stylC' 

Ui 

L database 7 by. a user (the customer or the like) is received by the controller 8: The 

yi 

. selected hair-Style data is read outfi-om the hair-style database 7; and is supplied to the 

U1 

B hair style mapping processing section 4. 

nj 

In this hair style selection, icons, thmnbnails, or the. like indicating hair styles: 
. are .displayed. on the display: device 9 to invite the usento make a selection: . The 
selection may be done by moving a cursor or the like on the screen of the display 
- : device 9 or by inputting: a number or the like designating a hair styJe . . 

: r - In the . step S4, the hair style mapping processing section .4 of the image . 
: .processing device maps. the supplied hair-style data on the three-dimensional mode of 
.the head part of the customer supplied from the three-dimensional head . part 
- reconstruction processing section 3, and outputs the result to the rendering. part 5. In 
the processing of mapping the hair-style data, hair data corresponding to each one of 
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hairs, which is included in the hair-style data, is mapped on the surface of the 
three-dimensional model of the head part, in accordance with the shape of the 
three-dimensional model of the head part, by the saine manner as taken for texture 
mapping. Details of the mapping processing will be described later with reference to 
FIGS. 5.and.6. . . . .. 

^ . : In the step .85, the rendering section 5 renders the threerdimensional model of 
p . the head part of the customer, on which the hair-style data, has been mapped, to 
03 generate hair-style-matched image data as shown in FIG. 4, and outputs it to the 

a 

W display device 9. The method of rendering adopts texture mapping processing for a 

yJ 

%^ . polygon model with respect to the face part of the three-dimensioiial model; and also 
. rendering processing for a curvature model with respect to the hair part. The rendering 
processing for a curvature model is disclosed in details in, for example, "Simulation 
based oil. a motion model of a string-like object and GG" by Kobayashi, Monshima, 
:: and Harashima, Shingakugihou, PRU90-127, pp. 15^20^:1991... . 

The display device- 9 displays an image corresponding to the inputted 
hair-style-matched image data. The controller 8 invites a user (acustomer or the like)- 
;to input whether or not the user is satisfied with the hair-style-matched image 
- displayed on the display device 9. 
.. . . At ..this . time,, the user can operate the ; controller . 8 ..to watch., the 
hair-style-matched image while changing the viewpoint position and the illumination . 
position. If the viewpoint position and the illumination position are changed, the 
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rendering section 5 executes again the rendering processing on the three-dimensional 
model of the head part and the hair-style data, so as to correspond to the changed 
condition (the viewpoint position and the illumination position) and outputs the result 
to the. display device 9. 

. . In the step S5, if an input indicating that the user is not satisfied is received, the 
processing returns to the step S3, and the later processing is repeated. In the step S5, 
if it is detemiined that an input indicating the user^s satisfaction is received from the 
user, this processing is terminated. 

Next, explanation will be made of details of the hair-style mapping processing 

. based on the hairrstyle mapping processing section 4..in the step S4.: . Beforehand, a^ 
structural example of the hair-style mapping processing- section 4 ..will be explained: 
with reference to FIG. 5. 

, : The hair-style data read outsecti 21 of the hairrstyle mapping processing 

: section 4 reads hairrstyle data selected with use of the controller 8, from the hairrstyle: 
database 7, and outputs it to the mapping position calculation section 22. The mapping 
position caleulation section 22-calculates the position on the. three-dimensional model 
wherehair data (corresponding to a hair) foraiing part of the hair-style data should be 
mapped. A hair data interpolation processing section 23 obtains hair data to be 
mapped, by interpolation from the peripheral hair, data around the hairrStyle data. The , 
coordinate conversion section 24 maps the interpolated hair, data -by coordinate 
conversion from the object coordinate system of the hair data to the object coordinate 
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system of the three-dimensional model of the head part. 

Next, details of the hair style mapping processing of the hair style mapping 
processing section 4 will be explained with reference to the flowchart shown in FIG, 
6. The mapping position calculation section 22 of the hairstyle mapping processing, 
section 4 has already. been supplied with hair-style data from the hair-styJe data read . 
outsection'21. . • / . 

. The hair-style data supplied to the mapping position calculation section 22 is 
obtained by two-dimensionally arranging hair data approximated to an object having 
a three-dimensional shape, as shown in FIG. 7. 

.. . ; . In.the . example of FIG. 7 A, every hair is approximated, to a special curve such 
as a three-dimensional B-spline curve or the like. In the example of FIG. 7B, eveiy^ 
hair is approximated to a polygon model. The addresses in the two-dimensional array 
correspond to mapping coordinate values supplied to the three-dimensional model of 
the head^part on which the hair style is mapped, like the texture mapping processing. 

For example, if a mapping coordinate (u, v) = (0. 1, 0.1) is given to a vertex of 
.a predetermined polygon of the three-dimensional model of the head party the. address 
on the hair-style data array of the hair data which.is mappedon the vertex is (0. lu^; ; 
0.1Vn,aJ. i AVhere, the u^^x the number of data pieces in the u-directipn on the 
hair-style, data array^,. and v^^^ is the number of data pieces in the v-direction on the 
hairrstyle data array... 

The hair-style data is defined by arranging the hair data having an arbitrary 
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three-dimensional shape into a two-dimensional array. Therefore, any arbitrary hair 
style can be expressed, and it is possible to generate hair-style, data of a hair style 
which is difficult to cut. The method of generating hair-style data will be described 
later with reference to FIG.. 16. 

At first, in.the step S I 1, the mapping position calculation section 22. initializes 
the count value v in the longitudinal direction of the hair-style data array to Q.O. In the 

step S12, the count value u in the lateral direction of the hair-style data array is. 

. . . . 
f^=5 initialized to 0.0. 

yj In the step S13, the mapping position calculation section 22 calculates the 

« 2 position on the three-dimensional model of the head part.where hair data expressed by 
r! the address (u, v) .of the hairrstyle data array is mapped. \ 

p Explanation will now be made more specifically. FIG. 8 shows an arbitrary 

F: S 

' . tri angle poly gon on the three-dimensional model of the head part. The vertexes A, B , 
and e of the triangle polygon respectively have attribute vectors A^t^, Bart;;i and Cattr 
which express attributes of the three-dimensional model of the head part. 

: The . attribute . is . constructed by an object coordinate value of the 
three-dimensional . model of the head part, local coordinate. axes at a vertex, and a 
... mapping coordinate value of hair. Where vectors which form two arbitrary edges, of 
: ..the triangle polygon ABC are vectors VI and V2, the vectors: Yl and V2. can be . 
expressed by the next expression (1) with use of the attribute vectors of respective 
vertexes. 
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V1=B.,^-A„^ 

V2 = C,^-A.^ ...(1) 
... Considering that the vectors VI and V2 are the coordinate axes of the ti iangle 
polygon ABC, an arbitrary point on the. triangle polygon ABC can be expressed by. 
the next. expression (2). 

Ph = A,^ + aVl+bV2...(2) 

At this time, as shown in FIG. 8, coefficients. a and b are defined in a manner, 
that lengths between an arbitrary point and points at each of which a line parallel 
to one axis and extended from the arbitrary point crosses the other axis are nonnalized 
by the lengths of axial . vectors, respectively. 

Where the mapping coordinate values of the vertexes A, B, and C are 
respectively (u^, vj, (Ub, v^), and (u^, vj, attention is paid to the point Ph expressed by 
the expression (2), in relation to the elements of the mappingcoordinate values. The 
next expression (3) then exists. v i .^r. : ^ : . . 

- -Uh-U. + aCUb + UJ + bCU.-UJ 
vv.vA^,.=:V, + a(Y,.-VjH-b(V,..VJ . ■ : ^ 

The expression (3) exists with respect to an arbitrary point in tfie (U,V) space. 

. If the coimter value u = Up and the counter value v = Vp, the. condition for .. 
. satisfying that the point Ph on the three-dimensional model, of the head part, where hair , 
data should be mapped exists inside the triangle polygon is expressed by the next 
expression (4). 
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a + b < 1 ...(4) 

That is, in the arbitrary triangle polygon of the three-dimensional model of the 
head part, if the coefficients a and b obtained by substituting u,, = Up and Vh -= Vp in the 
expression (3) satisfy the expression (4), the point Pj, exists inside the triangle polygon. 
Therefore, the hair data expressed by (Up, Vp) on the hair-style data array is mapped on 
the triangle polygon. Accordingly, the mapping position of the -hair, data on the 

Li 

. C3 three-dimensional model of the hair data is calculated by using the expression (2) in 

P 

^t, the mapped triangle polygon. 

yj Returning to FIG. 6, in the step S 14, the mapping position calculation section 

3 22 determines whether or not the triangle polygon of the three-dimensional model of 

b 

: Ul , the head part to be mapped is a hair generation region. This determination is earned 
^ out, based on a flag (positive or negative) which indicates whether or not hair data 
: : added to each triangle polygon of the three-dimensional model of the headpart should 
- Remapped. 

If the mapping flag of the triangle polygon including the mapping position 
obtained by the processing in the step S13 is positive, the triangle polygon is 
...... determined to be a hair generation region, and the processing goes to the step S 15.. 

In the step S 15, the hair data interpolation processing section 23 interpolates 
- . hair data^. which is expressed as (Up, Vp) and should be mapped, with use of hair data^ 
... stored in the hair-style data array. The interpolation method may use an average value 
of four close points or may use a summation average value of four close points. 
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Alternatively, the closest point may be directly used without interpolation. 

FIG. 9 shows a method of using a summation average value from four close 

. points. Supposing now that the counter value u = Up and the counter value v =^ Vp, the 

hairrstyle data array address of the hair data to be mapped is (mp. Hp) = (up .u,„a>c. 

Vp v,„„x)- The hair data at the point P„ is Hp, and the close 4 points are PI to.P4 as 

showninFIG. 9.. The hair data pieces at the point PI to P4 are respectively HI. to H4 

Q\ ■:: -In the summation average method from 4 close points, the hair data Hp to be 
p 

mapped is interpolated from the data of the 4 close points with use of the next 

i;1 expression (5:). ' . . : 

f- = . Hp^ dm (dn :H2-+.(l-dn) HI) + (1-dm) (dn H4 + (1-dn) H3=));.. (5) 
C3 

. UT v .. If the hair data is a. special curve such as a three-dimensional Brsplrne curve or : 

tt 

the like as shown in FIG. 7A, the interpolation processing of the hair data is obtained 

C5 

- by interpolation of data of each control point. Meanwhile, if theliair data is a polygon 
model as shoAvn in FIG. 7B, the interpolation processing of the hair ;data is obtained: 
by interpolation between respective polygon vertex data pieces; In the expression (4), 
dm is the decimal part of nip,, and dn is the decimalpart of np. • 
: : /Returning to FIG: 6^ in the step S16, the coordiniate. conversion section 24 

coordinate converts hair data subjected to interpolation; processing, from the object- 
•.i: coordinate, system of . the hair data into the. object., coordinate system of the. 
: : three-dimensional model of the head part. . . . . 

The concept of the coordinate conversion will be explained with reference to 
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FIGS. lOA and lOB. Suppose that an arbitrary hair data piece on the hair-style data 
array is H. The hair data H is expressed as (I, J, K) according to the object coordinate 
. system of the hair data. The object coordinate system (I, J, K) of the hair data may be 
. arbitrarily given in case of a three-dimensional coordinate system in which the start 
. point of the hair data H is an origin. For example, this is an orthogonal coordinate 
. . system, in which the lateral direction of the hair-style data array is I, the -longitudinal 
£3 - direction thereof is J, and a direction perpendicular to these directions is K.. Thus, the 

a 

i"^ Start point of the hair data H is expressed as (I, J, K) = (0, 0, 0), and the other control 

D3 

points are expressed by relative positions from the start point of the hair data. 

i . i 

: Meanwhile, the three-dimensional model of the head part is expressed as an 

m object coordinate system . (X, Y, Z). The object coordinate : system of the 

M 

yi three-dimensional model of the head part is generally a three-dimensional orthogonal 

a 

H- : coordinate system^ e.g. , ari orthogonal coordinate system in which the direction toward 
: the right ear from the gravity center position of the head part as an origin is X, the 
direction toward the top of the head is Y, and the direction toward the nose is Z. 
. : The coordinate conversion section 24 converts the hair data H expressed by. the 
.object coordinate system (I, J, K) of the hair data into a local coordinate system (T, S, 
. . N) at the mapping position of the three-dimensional model of the head part obtained 
■ in the step S 13. The local coordinate system (T, S, N) at this point can be arbitrarily 
determined in case of the three-dimensional coordinate system using the point P^ as 
an origin. For example, it is an orthogonal coordinate system in which N is the vector 
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in the nomial line direction at the point P,,, T is the vector in the tangent direction, 
which is . obtained as a vector product of the vector in the radial direction toward the 
point Ph from the center axis (Y-axis in FIG. 10) in the longitudinal direction of the 
three-dimensional model of the head part and the vector N in the nonxial line direction, 
and S is the. vector perpendicular to the nonnal line direction N and the. tangent 
directionT. . . - - - 

.Suppose thatthe local coordinate system at each vertex of the three-dimensional 
model of the head part is previously given in the head-part database. Therefore, like 
the calculation of the mapping position, the local coordinate system (T, S- N) at the 
point -P^ is calculated with use of the expression (2) in the triangle polygon to be 
mapped. : Supposing that data at the object coordinate system (I, J, K) of an arbitrary 
control point Cj of the hair data H is (i, j, k), the control point C, is expressed by the 
next expression (6) in^ the local coordinate system (T, S, ^N^^ 

Where, the unit vectors of the local coordinate system (T, S, N) respectively 
havervailties of the object coordinate system (X, Y, Z) of the three-dimensional model 
.of the. head part: : ' . , . : , ■ ; > : 

_ . Therefore, thexontrol point C; expressed by the expression (6) is expressed by 
•rthe.object.coordinate system (X, Y, Z) of the threerdimensional.model of the head 
part. ... . ...^ . 

Returning to FIG. 6, in the step 817, the mapping position calculation section 



25 

22 detennines whether or not the counter value u exceeds 1.0. If the counter value u 
is not determined to exceed 1 .0, the processing goes to the step S 1 8. In the step S 1 8, 
the mapping position calculation section 22 increase the. counter value u by an 
increment value du. The increment value du indicates.how long pitch hair data stored 
. in the hair-style, data array is mapped in the lateral direction. 

Thereafter, the processing returns to the step S13, and the later processing.is 
repeated. If it is determined in the step S 17 that the counter, value u exceeds 1.0, this 
means that mapping processing in the lateral direction reached the end of one line of 
the hair-style data array, so the processing goes to the step S I 9. ^ ' . . 

... -In the step S 19, the mapping position calculation section 22 determines whether 
:or not the counter value v exceeds 1.0. If the counter value v is detemiined as ..not 
exceeding 1.0, the processing goes to the step S20, In the step S20, the mapping 
position calculation section 22 increases the counter value v by an increment value dv. 
The increment value dv indicates how long pitch hair data stored in. the hairrstyle data 
array is mapped in the longitudinal direction. 

. : .Thereafter, the processing returns to the step S 12 and the later processing, is 
repeated. In the step S19, if the counter value v is determined as exceeding LO,. this 
means that mapping processing is completed with respect to. all lines. . The hair 
mapping.processing is terminated, and the processing. goes to the step S5 shown in 
FIG. 3. 

In the step S14, if the mapping flag of the triangle polygon including the 
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mapping position obtained by the processing in the step S13 is negative, the triangle 
polygon is not determined to be a hair generation region, and the processing of the 
steps S 15 and.S 16 is skipped. 

By. thus specifying a hair generation region with use of a. mapping flag, 
mapping can be effected only on an arbitrary region of the tliree-dimensional model . 
of the head part, without mapping all the hair data pieces that constiaict tlie hair-style 
. data read outfrom the hair-style database 7. Therefore, a hair-style such as a so-called 
Machican cut, a star cut, or the like can be easily generated. 

Also, according to this hair-style mapping processing • each of the hair-style data 
pieces stored in the hair-^style database 7 is expressed by the object coordinate system^ 
of hair and can be mapped on a three-dimensional model of a head part having any 
arbitrary shape, without depending on the shape of the hmnan head part. 

.Next, .the data, structure of the hair-style database 7 will be^xplained with 
reference, to FIG; 1 1 : . The hair-style database 7 is constructed by a plurality , of 
hair-style data pieces. Each hair-style data piece is of a two-dimensional array as 
deiscribed.abbve, and at the top, the number of data pieces Hj^^x in thelateral direction 
and the.riumber of data pieces v^^x the longitudinal direction. Following them, hair 
data pieces for (Umak ^ v^aJ described. 
- .... ... In some cases, group information (described later) is described at the top of the . 

hair data piece. Subsequently, a plurality of control point data pieces are described, 
in case where hair data is approximated to a three-dimensional parametric curve or the 
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like. The control point data pieces are expressed by the hair object coordinate system 
(I, J. K). 

The hair data may be three-dimensional polygon data. In this case, a plurality 
of polygon vertex.data pieces are described as hair data. . • 

. The group information described in hair data will be explained now. The group 
infomiation can be used for hair data interpolation processing in tlie .step:S15. of the 
hair-style mapping processing (FIG. 6) described above: 

With reference to FIG. 12, explanation will be made of the hair data 
interpolation processing in case where group information is supplied. Where the 
counter value u:=: Up and the counter value v = Vp are given,, the address on the hair- 
style data array of hair data to be mapped is =^ (mp, np)..=^ (u^ ^iiax»"v:p v^ax). 

Suppose that the hair data at the point is Hp, four close points are points PI 
to P4 as shown.in FIG. 12, ahdhair data pieces at the points PI to P4 are respectively . 
Hl;to:H4. Further^. 1 is described as group information in the hair:datapiece HI, and 
2 is described as group information in each of the hair data pieces H2 to H4 . That is, 
in.this exaniplei only the hair data piece HI belongs to a group Iv and the other hair 
data pieces H2 to H4 belong to a group 2. If group information is given, only the hair 
data pieces that belong to one same group are used to perform interpolation, in the hair 
data interpolation processing. ^ z.:-.:^: . ... c.. . 

- More specifically, among the four close points of the point P^. which is mapped 
at first, the closest point if obtained and the group information thereof is obtained. In 
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case of FIG. 12, the closest point is the point P3. Next, among the four close points 
PI to P4, the number of close points that belong to the same group as that of the 
. closest point is obtained. 

If no other point among the four close points PI to P4 than the. close point 
. belongs to the same group as that of the closest point,. the.hair data piece of thexlosest 
pointis directly used also for the point Pg. 

If two close points including the closest point among the four close points P I: i 

03 to P4 belong to the same group as that of the closest point, a linear interpolation value 

C3 

yj between the two points is used for interpolation. 

UJ 

3 . .. Ifthree close points including the closest point among the four close points PI . 

Cj 

to P4 belong to the same group as that of the closest point, i.e. ^ in the case shown in 
n FIG. 12, an interpolation value is calculated by the next expression (7). 

-Hp.^^L-r)H32 + rH34 . -5 - 

A. -: -H32^=f dh H2 + (l.dn)H3- " \ ^ 

= H34 = dnH4 + (l-dn)H3 

:r:=(mp.-rn32)/(m34-m32)... (7) . . 

. . • . - v. In this expression, dn is the decimal part of nj,, m32 is the.m-address at the point : 
P32 ■ and m34 is the m.-address at the point P34. . ^ . - 

... By thus describing group information in hair data, for example, it is possible to - 
express even a hair style having parting of hair where interpolation will cause a 
problem. That is, if grouping at the botmdary of parting is performed, hair data pieces 
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in both sides of the boundary can be used to prevent generation of unnatural hair, in 
case where four close points spread over the boundary. 

Next, with reference to FIGS. 12 and 13, explanation will be made of an 
applicative example of the coordinate conversion in the processing in the step S I 6 in 
the hair-style mapping processing (FIG. 6) described.above. 

In. the coordinate conversion processing in the step SI 6, unevenness in a hair 
style can be expressed with use of random numbers. FIG. 13 A shows a state where 
unevenness in a hair style is not expressed while FIG. 13B shows an example of a state 
in which unevenness in a hair style is expressed by rotating the local coordinate system 
.with use of a random number in the coordinate conversion. . . 

/: Specifically^ the local coordinate system (T, S, N) of the'head part is. rotated J 
about an axis in correspondence with a generated random nmnber where the axis is N. 
Thereafter, the result is converted into the object coordinate system (X, Y, Z) of the 
head part model. By this processing, the directions in which respective hairs grow can 
be changed delicately, without changing the positions where hair data should be 
mapped. : ; 

: FIG. :14A'shows a state in which unevenness in a hair style is.not expressed, 
while FIG. 14B shows an example in which unevenness in a hair style is expressed by 
changing mapping positions by random numbers in- the. coordinate conversion. .. . 

. Specifically^ the position of as the origin of the local coordinate system (T, 
S, N) is shifted to P„ in correspondence with a generated random number, and is 
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thereafter converted into the object coordinate system (X, Y, Z) of the head part 
model. By this processing, the position where the hair grows can be changed 
delicately without changing the hair style. 

The technique of rotating the axis of the coordinate system to be con verted. use 
the random number and/or moving its origin to. express natural, unevenness in a 
synthesized^image can be applied to other cases than the synthesis of hair images. 

. . Meanwhile,. in this hair synthesis system, hair-style data can be mapped. after 
changing the three-dimensional model of the head part into an arbitrary shape by the 
three-dimensional head part reconstmction processing section 3. FIG. 15A shows a 
state in which the three-dimensional model of a head part is not changed, . while FIG. . 
15B shows a state in which the three-dimensiond model of. the head part^^ ^ 
1.5 times in the Y-axis direction. 

Magnification/minification processing for a three-dimensional model of a head 
part is achieved , by affine conversion processing. At this time, similar affine . 
conversion processing is performed also on the local coordinate system (T, S; N) 
assigned to an arbitrary polygon vertex of the three-dimensional model of a head part. 

. As shown in FIG. 15B, in case of magnifying the three-dimensional model of 
the head part for 1 .5 times in the Y-axis direction, vectors of T, S, and N are multiplied 
by a^conversion matrix for 1.5 times magnification in the Yraxis. direction, to obtain. , 
a local coordinate system (T\ S\ N') expanded in the longitudinal direction. By 
mapping hair data H on the local coordinate system (T', S\ N') thus deformed, the hair 
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Style is magnified 1.5 times in the Y-axis direction, into a shape matched with the head 
part. By this processing, same hair-style data can be mapped on the. three-dimensional 
model of a head part, which is defonned into an arbitrary shape. Next, generation 
, . of hairrstyle data will be explained with reference to. FIG.: 16. FIG. .16 shows a 
columnar, coordinate system which surrounds a head part. In the columnar coordinate 
system, the axis penetrating the center of a column is taken as the Y-axis, the distance 

p . fi-om the Y-axis to a plane perpendicular to the Y-axis is r, and .the angle frpm a 

i^^ reference direction on this plane is 6. 

m 

, . - An arbitrary point on the surface of the head part can be expressed as Ph = 

ft ^ (yp, r- 6p) with use of the columnar coordinate system. Further^ if the line defined by 
ill : Y = yj, and 0 =^ 0p of the columnar coordinate -system always has one contact point . 
yi with the surface of the head part, the arbitrary point P^ on the surface of the head part 
I -xaii.be: expressed by (yp, 6p). -..-r''-' -.v^. ^..-^ •J 

i That is, if the shape of the surface of the head part is always convex toward the . 
circumferential direction, the above condition (the line defined by Y= yp and 6 = 6p 
of the columnar coordinate system always has one contact point with, the surface of the 
. : : / head part); is satisfied. . = , ; - 

.... . - . Considering the . scalp part excluding the face, the shape of the surface of the. 
. . . . head .part is generally considered -to be always , convex, toward the circumferential. . 
.. . direction. Therefore,, the arbitrary point P^ on the scalp part of the head-part model 
can be expressed by P^, = (yp, 6p). 
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Creation of hair-style data can be relatively easily carried out by using a system 
disclosed in "Development of a hair design system using tuft models" by Kishi, 
. Saegusa, and Morishima, in Shingakugihou, MVE97-102, pp. 67^74, 1998. In the 
system, hair-style data is prepared by editing hair data generated on a given head-part 
model. Hair data outputted from this system is expressed by the object coordinate 
: . system of the given head-part model. 
O ..... : . The hair database 7 in the present invention records hair-style data which is 
^! stored in a two-dimensional array by mapping the hair data on a space expressed by 
yj (y^ ®) ^ columnar coordinate system. For example, the following expression (8) is 

yj 

B used forxonversion from the'(y, 6) space into the (u,.v) space of the two-dimensional 

C3 

Ul ; array as shown in FIG.^. : - v^..^;:; 

y] u=e„/27r 

ni 

.. v-(yp-y^i„)/(y^,x-y^J...(8) 

: In. this expression, y^i„: is the smallest value of the Y-^axis iof the columnar 
coordinate system of the surface of the head part, and y^^^ is the largest value of the 
. .Y-spcisrof the columnar coordinate system of the surface of the head part. . . - . 

The conversion from the (y, 6) space into the (u, v) space may be of non-linear. 
For example, sampling near the top part of the head can . be. prevented from. being. 
. coarse, as long as the, conversion is arranged such that.the.increment in the v-direction 
.becomes smaller as it. comes closer to the head top. . ^ 

As described above, hair-style data consisting of hair data converted into the (u. 
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v) space is recorded in the hair-style database 7. 

The present invention is also appHcable to the case of expressing a string-Hke 
object other than hair in GG. 

Meanwhile, the . series of processing in the image processing apparatus 2 as 
described above can be executed by both hardware and software. In case of executing 
the series of .processing by software, the software.can.be installed fi"om a recording, 
rj. medium into a computer . in which the program constituting, the : software is 

5 

1=^ incorporated in dedicated hardware or a general-purpose computer which can execute 

m 

various ftmctions computer by installing various programs. - 
^' . . . . . .TIG. . 17; shows a striictural example of a personal computer which realizes the. 

\f\ image processing apparatus 2. This personal computer includes a CPU (Central 

u 

Ul Processing Unit) 41. The CPU 41 is connected with an input/output interface 45 
^- through a bus 44. The input/output interface 45 is connected with: an input section 46 
cdnstmctedby.an input .device such as a keyboard, a mousey or the like; ah :iinage . 
output section 47 for outputting a hair-style-matched image to the display device 9; a 
( storage section 48 constructed by a hard-disc drive for storing a program, various data, 
. . . and the. like- a communication section 49 for communicating data through Internet or 
.the like; and a drive 50 for reading/writing data with respect to a recording medium . 
. such as a magnetic disc 51, a semiconductor memory.54, or. the like.. The bus 44.is- 
connected with, a ROM (Read Only Memory) 42 and a RAM (Random Access 
Memory) 43. 
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An image processing program which lets this personal computer execute 
operation as the image processing apparatus 2 is supplied in the fonn that it is stored 
in the magnetic disc 5 1 or semiconductor memory 54, to the personal computer. It is 
read by the drive 50 and installed in the hard disc drive included in the-storage section 
. , 48. The image processing program installed in the storage section 48 is loaded from 
J the storage, section 48 to.the RAM 43 and executed, in accordance with instructions 

pis 

n . .from the GPU 41 corresponding to conimands from t^^ . . 

03 In the present specification, the step of describing the program recorded in the 

a 

yj recording medium includes not only processing which is carried out based on time in 

yj 

accordance with.a described order but also processing which is executed in parallel or 
u - individually although itis not always performed based on time. v 

yi 

Also, in the present specification, the term of "system" means the entire 

nJ 

. ^ apparatus constructed by a plurality of apparatuses and/or devices. . ; ■ ■j^ . / . . 

Industrial Applicability 

; -As described above, according to the present invention, the hair-style is, stored, - 
. stored hair-^style data is read out, and hair data corresponding to a hair and included 
. in the hair-style data is mapped at a predetermined position of a three-dimensional 
. shape image of a head part. . .Therefore, a hair style consisting of hairs every of which 
. . is expressed by CG can be changed so as to correspond to plural different head shapes. 
Also, according to the present invention, stored hair-style data is read out, and 
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hair data corresponding to a hair and included in the read outhair-style data is mapped 
at a predetermined position on a three-dimensional shape image of a head part. Based 
on the mapping results, a hair-style-matched image is generated. Therefore, a hair 
style consisting of hairs every of which is expressed by CG can be changed so as to 
correspond to plural different head shapes. 



